In order to provide scalability in the size and bandwidth of a distributed file system, conventional storage systems have spread the storage of files across a plurality of storage nodes in a distributed storage system. Conventionally, while the storage of files in storage nodes has been dispersed, the management of the file system, processing and file metadata have been centralized in a control node or task management node.
In conventional distributed file systems, methods for locating data within the file system have used mechanisms which are considered “out-of-band” or unsupported by the protocols typically used to access the file system. One example of an out-of-band mechanism is used in the ParaScale™ distributed storage platform. In a ParaScale™ distributed storage platform, a centralized task manager or master, opened a file over a mount point exported via Network File System (NFS), obtained key information about the file using a “stat” operation, and then communicated a data location service on a control node using the file's key information, a received Extensible Markup Language (XML) information of the location of data, such as a server's address. A client of the ParaScale™ platform could then launch a data task with the server's address.
As noted above, conventional distributed file systems use a centralized task manager which is responsible for coordinating tasks on all nodes. Typically, the task manager directs the other nodes to execute tasks. While such a configuration allows for tasks to be performed in a known order, if the task manager fails, the resiliency of the distributed file system will suffer since the task manager must recover from the failure before additional tasks can be launched. Furthermore, a log of pending tasks has conventionally been placed on shared persistent storage or replicated between multiple nodes. As such, the task manager poses a problem of being a single point of failure for the distributed file system.
In distributed systems, IP address assignment to nodes in the system has conventionally been handled by Dynamic Host Configuration Protocol (DHCP). If virtual machines are present in the distributed system, DHCP poses a problem in that DHCP generally assigns IP addresses to all systems in the network. As a result, problems exist in DHCP with respect to configuring a DHCP server which can be readily isolated to provide IP addresses to only the virtual machines present in the distributed system.